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Abstract 

Background: IK recent paper reported the outcomes of a study examining a new self-guided internet-delivered treatment, 
the Wellbeing Course, for symptoms of anxiety or depression. This study found the intervention resulted in significant 
symptom reductions. It also found that automated emails increased treatment completion and clinical improvements in a 
subsample with elevated anxiety and depression. 

Aims:lo examine the clinical outcomes and the effect of automated emails at 12 months post-treatment. 

Method: Participants, who were randomly allocated to a Treatment Plus Automated Emails Group (TEG; n = 1 00), a standard 
Treatment Group (TG; n = 106) or delayed-treatment Waitlist Control Group (Control; n = 51), were followed up at 12 months 
post-treatment. Eighty-one percent, 78% and 87% of participants in the TEG, TG and treated Waitlist Control Group 
provided symptom data at 12-month follow-up, respectively. The primary outcome measures were the Patient Health 
Questionnaire-9 Item Scale (PHQ-9) and the Generalized Anxiety Disorder-7 Item Scale (GAD-7). 

Results: Significant improvements in symptoms of anxiety and depression were observed over time in both the TEG and TG 
(Fs >69, ps <.001) these were sustained from post-treatment to 12-month follow-up (ps >.05), and were associated with 
large effect sizes. No statistically significant differences in symptoms were found between the TEG and TG at post-treatment, 
3-month or 12-month follow-up. Previously reported symptom differences between TEG and TG participants with comorbid 
symptoms were no longer present at 12-month follow-up (ps >.70). 

Conclusions: The overall benefits of the Wellbeing Course were sustained at 12-month follow-up. Although automated 
emails facilitated Course completion and reductions in symptoms for participants with comorbid anxiety and depression 
from pre-post treatment, these differences were no longer observed at 12-month follow-up. The results indicate that 
automated emails promote more rapid treatment response for people with elevated and comorbid symptoms, but may not 
improve longer term outcomes. 
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Introduction 

Self-guided internet-delivered psychological interventions for 
anxiety and depressive disorders have considerable public health 
potential. However, more than 90% of consumers typically 
withdraw after two sessions [1,2] and, consequently may not 



receive a dose of therapeutic content sufficient to lead to sustained 
improvement. Self-guided internet interventions that include 
strategies to facilitate adherence have considerable potential for 
optimizing outcomes. Trials of self-guided internet interventions 
for depression [3,4] and social phobia [5,6,7] show that automated 
reminder emails increased completion rates and effect sizes relative 



PLOS ONE | www.plosone.org 



1 



February 2014 | Volume 9 | Issue 2 | e89591 



The 12-Month Follow-Up of Self-Guided Treatment 



to earlier trials without automated reminders. Automated emails 
may be particularly suitable as a medium for providing reminders 
given their ubiquity and low cost. 

A recent paper reported an evaluation of a new self-guided 
internet-delivered treatment, the Wellbeing Course, for symptoms of 
anxiety or depression [8]. Participants had no contact with 
clinicians prior to or during treatment and were randomly 
allocated to receive the Wellbeing Course administered with 
automated emails (Treatment Plus Automated Email Group: 
TEG, n = 100), without automated emails (Treatment Group: TG, 
n= 106) or to a Waidist Control Group (Control Group; n = 51), 
who subsequendy received treatment. At post-treatment, partic- 
ipants in the two Treatment Groups obtained similar and 
significant improvements on measures of anxiety and depression 
relative to the Control Group. However, those participants 
receiving automated emails obtained higher rates of course 
completion, defined as having read all five lessons of the course, 
than those not receiving emails (58% v. 35%). While no overall 
differences in symptom improvement were found between the 
Treatment Groups, a subgroup of participants with comorbid 
anxiety and depression, that is, participants with elevated 
symptoms of both anxiety and depression, obtained superior 
reductions in symptoms when they received automated emails 
compared to when they did not receive the emails. The pattern of 
these results was sustained at 3-month follow-up. 

An outstanding question is whether these results are sustained in 
the longer term. Return or exacerbation of symptoms may occur 
following treatment for anxiety disorders and depression [9,10]. 
The primary aim of the present study was to test the hypothesis 
that the benefits observed at post-treatment and 3-month follow- 
up would be maintained at 12-month follow-up. Previous results 
indicate that benefits of guided internet interventions for anxiety 
or depression [11,12,13] are sustained in the longer-term, but 
limited data exist for longer-term benefits of self-guided internet 
interventions [14]. An additional aim was to examine the number 
of participants who deteriorated at 1 2-month follow-up, defined as 
an increase in PHO_-9 or GAD-7 scores of 5 or more points from 
pre-treatment. No formal hypothesis was generated with respect to 
deterioration, but this data was collected to provide information 
about the safety of the Wellbeing Course. 

Methods 

Ethics Statement and Trial Registration 

The original study [8] was approved by the Human Research 
Ethics Committee (HREC) of the University of New South Wales 
(UNSW, Sydney, Australia) and ratified by the Macquarie 
University HREC (Sydney, Australia; HREC Reference: 
5201 100226). The original protocol included telephone interviews 
at recruitment and a fourth group who received weekly contact 
with a therapist. However, in order to focus on the benefits of 
providing automatic emails during self-guided treatment and to 
simulate an entirely automated treatment process, the interviews 
and this fourth group were omitted from the final design. The trial 
was registered with the Australian and New Zealand Clinical 
Trials registry as ACTRN 126 1000 1058066. The protocol for this 
trial and supporting CONSORT checklist are available as 
supporting information; see Checklist SI and Protocol SI. 

Design and Sample Size 

The original trial randomly allocated 257 participants to one of 
two Treatment Groups or to a Waidist Control Group. 
Participants in the Treatment Plus Automated Email Group 
(TEG; n = 1 00) received access to the Wellbeing Course with 



automated emails. Participants in the Treatment Group (TG; 
n=106) received access to the Wellbeing Course without 
automated emails. The Control Group (Controls; n = 51) received 
access to the same treatment as the TEG following post-treatment; 
the data presented here are the data for the Control Group in 
Treatment, that is, after the Two Treatment Groups completed 
treatment and the Control Group received treatment. 

Measures 

The primary outcome measures were depression and anxiety 
symptom severity, as measured by the PHQ_-9 and GAD-7. The 
GAD-7 has good convergent validity with other anxiety scales 
[15-17], is sensitive to DSM-IV congruent generalised anxiety 
disorder, social phobia and panic disorder, with increasing scores 
indicating greater symptom severity [16]. A total score of 8 on the 
GAD-7 has been identified as an important threshold for 
identifying the presence of an anxiety disorder [15-17]. A total 
score of 10 on the PHQ_-9 has been identified as an important 
threshold for identifying DSM-IV congruent depression with 
increasing scores indicating greater symptom severity [15,17-20]. 
Additional information collected comprised demographic details 
including: age, gender, educational qualifications, vocational 
status, treatment satisfaction and completion rates (i.e., percentage 
in each treatment group who read all five online lessons of the 
Wellbeing Course). Additional measures included changes in 
health status and service utilisation, which will be reported 
separately as part of an evaluation of the cost-effectiveness of the 
intervention. 

TEG and TG participants completed the PHQ;9 and GAD-7 at 
pre-treatment, prior to each of the five lessons, at post-treatment, 
3-month follow-up and 12-month follow-up. Control Group 
participants completed the PHO_-9 and GAD-7 at pre-treatment 
and post-treatment and, subsequently, commenced the same 
intervention as the TEG group with data collected at pre- 
treatment, prior to each lesson, at post-treatment, 3 month follow- 
up and 12 month follow-up. Their results after they received and 
completed the treatment phase are presented here as a partial 
replication in order to test the reliability of the TEG condition. 

Intervention 

The Wellbeing Course is a five lesson transdiagnostic internet 
intervention based on a pragmatic model of psychotherapeutic 
change that assumes that symptoms of anxiety and depression are 
the result of unhelpful habits of thought and action, that is, 
maladaptive cognitions and behaviours [8,22]. This model also 
proposes that interventions that are structured, systematically 
present increasingly complex material over weeks and months, and 
are sufficiently engaging to achieve adherence and commitment, 
will result in reductions in unhelpful habits and an increase in 
adaptive habits. Further, such interventions are likely to result in 
large and sustained improvements in symptoms compared to 
sporadic or unstructured therapy sessions, which may only result 
in short-term symptom relief. More details about the Wellbeing 
Course can be found elsewhere [8]. 

Procedure 

TEG and TG participants received an email at the start of the 
Course providing guidelines and a recommended timetable of 
Course-related activities. TEG participants also received at least 
two additional emails per week during the Course. Emails were 
triggered when (1) participants read a Lesson, (2) if participants 
had not read a prescribed Lesson within 7 days of it becoming 
available, (3) at the start of each week, and (4) at times during the 
Course when participants were known to experience increased 
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symptoms or to have increased difficulties practicing skills (e.g., 
during the weeks when graded exposure was introduced). Emails 
were comprised of three or four sentences, used the participant's 
first name, and employed a warm and supportive tone. The 
content informed participants about newly available or unread 
materials, reinforced progress and skills practice, normalised the 
challenges of learning new skills, and encouraged realistic 
expectations of recovery. All participants consented to receive 
the emails and no emails contained personal or detailed clinical 
information. 

Up to five automated email reminders were sent to all 
participants to complete post-treatment or follow-up question- 
naires. These emails were only sent where participants had not 
completed the questionnaires. Once participants had completed 
the questionnaires they were sent a brief template-based feedback 
email detailing their results. At the 3- and 12-month follow-up 
time points participants who had not completed the questionnaires 
after the five automated emails were telephoned once by the 
researchers. No clinical contact was made between the 3- and 12- 
month follow-up periods and no clinical content was discussed in 
these telephone calls. The Course material was made available for 
the 1 2-month period following the end of the 8-week treatment. 

Statistical methods 

Consistent with the initial report of this trial [8], several separate 
mixed-models analyses were employed to analyse the data at post- 
treatment, 3-month and 12-month follow-up. These mixed-models 
employed an autoregressive covariance structure and maximum 
likelihood estimation, which provides unbiased estimates in the 
case of missing data; under the assumption that data is missing at 
random. These mixed-models specified Group (i.e., TEG and TG) 
and Time (i.e., pre-treatment, post-treatment, 3-month follow-up 
and 12-month follow-up) as fixed effects with the Intercept set at 
pre-treatment and Participants specific as a random effect. 
Primary interest was the Group x Time interactions and 
differences within and between the two Treatment Groups at 
the different time points were examined using pairwise compar- 
isons. Consistent with the original report of this study [8], one set 
of analyses was conducted using the entire data set (i.e., the Overall 
Sample) and a second set of analyses was restricted to participants 
with elevated symptoms of both anxiety and depression (i.e., the 
Comorbid Sample), defined by pre-treatment scores above clinical 
cut-offs on both primary outcome measures (i.e., total GAD-7 ^8 
and total PHQ.-9 >10). Effect sizes (Cohen's a) and 95% 
confidence intervals were calculated for both within-group and 
between-group effects based on the pooled standard deviation. All 
analyses were performed in SPSS version 19.0 (SPSS, Inc., 
Chicago, IL). 

As described in recent dissemination studies [23], pre-treatment, 
post-treatment and follow-up PHQ_-9 and GAD-7 scores were 
compared with clinical cut-offs to provide an index of clinically 
significant remission. This was defined as the proportion of 
participants who initially scored at or above the clinical cut-offs 
and then subsequendy below the clinical cut-offs. Finally, the 
proportion of participants with significant deterioration in PHQ;9 
or GAD-7 scores were reported, calculated for each group based 
on a completer analysis and defined as an increase in PHQ,-9 or 
GAD-7 scores of five or more points from pre-treatment [15]. In 
these analyses, missing values were handled using the Multiple 
Imputation [21] procedure where Group as well as GAD-7 and 
PHQ.-9 scores at each time point were used to generate 20 sets of 
imputed data. These 20 imputed data sets where then averaged to 
provide pooled frequency counts, chi-square statistics and p values. 
All results are reported also presented for the Waitlist Control 



Group, after they had received treatment, as a partial replication 
of the results of the TEG group. 

Results 

Adherence and Attrition 

Twelve-month follow-up data were collected from 81%, 78% 
and 87% of TEG, TG and treated Waitlist Control Group 
participants, respectively. 

Between post-treatment and follow-up participants in both 
treatment groups continued to log in and read the online 
materials. At the 12-month follow-up there was no longer a 
significant difference in the number of TEG (68%) and TG (59%) 
participants who had read the five lessons, that is, completed the 
Course, X 2 (1) = 1.63,/)= .20. Similarly, in the Comorbid Sample, 
no significant difference was found between the TEG (66%) and 
TG (59%) participants who had completed the Course at 12- 
month follow-up, X 2 (1) =0.67, p = Al. 

Primary Outcome Measures 

Means for the PHQ_-9 and GAD-7 at each time point are 
included in Table 1 and Figure 1 for the Overall Sample. Means 
for the Comorbid Sample are shown in Table 2 and Figure 1. 

Overall Sample. The mixed-model analyses examining 
GAD-7 scores for the Overall Sample revealed a significant effect 
for Time (F ? 467 = 87.57, /><.001), but not for Group (F t> 
2/2 = 0.11, /j = .74) or the Time by Group interaction [Fg 
4S? = 1.27, p = .28). Pairwise comparisons revealed no significant 
differences between the TEG and TG groups at post-treatment, 3- 
month follow-up or 12-month follow-up (p range =.14 to .69). 
Both the TEG and TG groups improved significantly from pre- 
treatment to post-treatment (/><.001), but did not obtain any 
further significant changes from post-treatment to 3-month follow- 
up (p>.05) or from 3-month follow-up to 12 month follow-up 
(fi>.05). However, the scores of both groups remained significandy 
lower at 3- and 1 2-month follow-up compared with pre-treatment 
(/><.001), indicating that symptom improvements were main- 
tained. 

Similarly, the mixed-model analyses examining PHQj9 scores 
for the Overall Sample revealed a significant effect for Time {Fg 
471 = 74.48, p<.Q0\), but not for Group {F h 219 = 0.01, P= .93) or 
the Time by Group interaction (F 3> 471 = 0.79, p= .50). Pairwise 
comparisons revealed no significant differences between the TEG 
and TG groups at post-treatment, 3-month follow-up, and 12- 
month follow-up (J> range = .35 to .81). However, both the TEG 
and TG groups improved significandy from pre-treatment to post- 
treatment (/)<.001), but did not obtain any further significant 
changes from post-treatment to 3-month follow-up (p>.05) or 
from 3-month follow-up to 12-month follow-up (/>>.05). The 
scores of both groups were sustained and remained significandy 
lower at 3- and 1 2-month follow-up compared with pre-treatment 
(p<.00\). 

Comorbid Sample. The mixed-model analyses for the 
GAD-7 scores for the Comorbid Sample revealed a significant 
effect for Time (F % 258 = 69.03, jft<.00 1) and Group (F h 126 =4.21, 
/> = .042), but not the Time by Group interaction (_F ; 25li = 1-63, 
/; = .18). Pairwise comparisons revealed that both the TEG and 
TG groups improved significandy from pre-treatment to post- 
treatment (/)<.001), but did not obtain any further significant 
changes from post-treatment to 3-month follow-up (/>>.05) or 
from 3-month follow-up to 12-month follow-up (p>.05). The 
scores of both groups were sustained and remained significandy 
lower at 3- and 1 2-month follow-up compared with pre-treatment 
(/>>.001). Pairwise comparisons also revealed the TEG group had 



PLOS ONE | www.plosone.org 



3 



February 2014 | Volume 9 | Issue 2 | e89591 



The 12-Month Follow-Up of Self-Guided Treatment 



GAD-7 Scores for the Overall 




Pre Post 3 Month 12 Month 

Follow-up Follow-up 



GAD-7 Scores for the Comorbid Sample 

16 -| — 
14 




2 

0 

Pre Post Follow-up 12 Month 

Follow-up 



PHQ-9 Scores for the Overall Sample 

14 n 




4 

2 

0 -I — — i — — i — — i — 

Pre Post 3 Month 12 Month 

Follow-up Follow-up 



PHQ-9 Scores for the Comorbid Sample 




Pre Post Follow-up 12 Month 

Follow-up 



Figure 1. Means for the PHQ-9 and GAD-7 for the Overall and Comorbid Samples. Note: TEG: Treatment plus automated email group. TG: 
Treatment group. PHQ-9: Patient Health Questionnaire-9 Item. GAD-7: Generalised Anxiety Disorder-7 Item. 
doi:10.1371/journal.pone.0089591.g001 



significantly lower scores than the TG group at post-treatment and 
3-month follow-up (p range =.01 to .03), but not at 12-month 
follow-up (p= .75). 

The mixed-model analyses of the PHQ_-9 scores for the 
Comorbid Sample revealed a significant effect for Time (Fg 
268 = 72.58, p<.Q0 1), but not for Group (F 1> 1M = 2.24, p = .14) or 
the Time by Group interaction (F 3i !e g = 2.27 > p= .08). Pairwise 
comparisons revealed that both the TEG and TG groups 
improved significantly from pre-treatment to post-treatment 
(/><.001), but did not obtain any further significant changes from 
post-treatment to 3-month follow-up (p>.05) or from 3-month 
follow-up to 12-month follow-up (p>.05). The scores of both 
groups were sustained and remained significantly lower at 3- and 
12-month follow-up compared with pre-treatment (p>.001). 
Pairwise comparisons also revealed the TEG group had signif- 
icantly lower scores than the TG group at post-treatment and 3- 
month follow-up (p range =.02 to .04), but not at 12-month 
follow-up (£ = .99). 

Waitlist Control Group (following treatment). The 
mixed-model analyses examining the scores of those in the 
Waitlist Control Group following treatment revealed significant 
effects for Time on both the GAD-7 (F 3i g5 . 38 = 12.80, p<. 001) and 
PHQj-9 (F 3 97.99= 15.71, p<. 001). Pairwise comparisons revealed 
significant improvements on both the PHQ_-9 and GAD-7 from 
pre-treatment to post-treatment (p<.00\), but no significant 
changes from post-treatment to 3-month follow-up (p>.05) or 
from 3-month follow-up to 12-month follow-up (p>.05). However, 



3-month follow-up and 12-month follow-up scores on the PHQ;9 
and GAD-7 remained significantly lower than pre-treatment 
scores (/><.001), indicating that the treatment effects were 
maintained. 

Effect Sizes 

Within- and between-group effect sizes for the outcome 
measures are included in Tables 1 and 2. For both the Overall 
and Comorbid Samples large within-group effect sizes (i.e., S0.80) 
were found for the TEG and moderate-to-large within-group (i.e., 
^0.50) effect sizes were found for the TG. Similar large within- 
group effect sizes were observed in the Control Group following 
treatment. Small and non-significant between-group effect sizes 
(i.e., £0.30) were found between the two Treatment Groups. 
Small-to-moderate between-group effect sizes were found between 
the Comorbid Samples (i.e., 0.30 to 0.50) at each of the time 
points, except at 12-month follow-up time point, when the effect 
sizes were small and non-significant (i.e., £0.30). 

Clinical Significance 

Overall Sample. As shown in Table 3, post-treatment 
remission rates for the PHQ_-9 and GAD-7 were sustained at the 
12-month foUow-up time points for the TEG (64.7% and 63.1%) 
and TG (68.6% and 60.8%). 

Comorbid Sample. As shown in Table 3, the post-treatment 
remission rates were also sustained at 12-month follow-up for the 
Comorbid Samples of the TEG (73%) and TG (74%). 
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Waitlist Control Group (following treatment). The re- 
mission rates for the Control Group and the remission rates for the 
Comorbid Sample of the Waitlist Control Group following 
treatment were also sustained at 12-month follow-up. 

Deterioration 

Overall Sample. At 12-month follow-up, up to 3.3% of TEG 
and 5.0% of TG participants obtained PHQ_-9 or GAD-7 scores 
five or more points higher compared to pre-treatment. No Waitlist 
Control Group participants had scores five or more points higher 
after treatment at 12-month follow-up. 

Comorbid Sample. At 12-month follow-up, 3.0% of TEG 
and 1.8% of TG participants in the Comorbid Samples obtained 
PHQ;9 or GAD-7 scores five or more points higher compared to 
pre-treatment. 

Discussion 

The primary finding from this study was that the self-guided 
Wellbeing Course resulted in clinically significant reductions in 
symptoms of anxiety and depression at post-treatment and these 
reductions were sustained at the 3- and 12-month follow-up time 
points. These findings are consistent with results from clinician- 
guided internet interventions, which demonstrate that benefits are 
sustained in the longer term [11,12] indicating the robust nature of 
internet interventions. The magnitude of clinical benefits from this 
self-guided intervention at 12-month follow-up were larger than 
those reported at follow-up than other self-guided internet 
treatments for depression or anxiety [12,24]. Benefits from the 
present trial also compare favourably with self-guided internet 
treatments for depression or anxiety where participants had 
contact with researchers at pre-treatment [25,26], with trials of 
transdiagnostic internet interventions involving clinician guidance 
[27,28,29] and trials of clinician-guided transdiagnostic interven- 
tions administered face-to-face [30]. However, while the magni- 
tude of gains reported here are encouraging, caution should be 
exercised in making comparisons across studies due to differences 
in samples, inclusion criteria, and methodologies. 

An important secondary finding of the present study was that 
the benefits of automated emails, in increasing completion rates 
and in reducing symptoms in a subsample with elevated and 
comorbid symptoms, were no longer found at 12-month follow-up. 
In the present study, at 12-month follow-up, the TG obtained 
similar outcomes to the TEG on both the PHQ;9 and GAD-7 and 
both groups evidenced remission rates above 50% at 12-month 
follow-up in each group. Rates of deterioration at 12-month 
follow-up appeared larger in the TG, but the overall event rate 
was low (<5%). 

The absence of benefits from automated emails appears to have 
been the result of continued improvements in the TG group, of 
whom an additional 25% completed the Course between the 3- 
and 12-month follow-up points; compared to 7% of the TEG 
group. Consequently, at 12-month follow-up, there was no 
difference between groups in the proportion who had read all 
five lessons or in the clinical outcomes of either the Overall or 
Comorbid Samples. The reasons for this increase in completion 
rates in the TG are unclear. However, it is possible that the 
automated assessment emails at post-treatment and 3-month 
follow-up prompted TG participants to return to the Course, 
although it is also likely that participants found the material in the 
Course sufficiently helpful in the first instance to justify their 
return. 

These results raise the question of whether automated emails 
are helpful or necessary. Our findings suggest that automated 
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Table 3. Proportion of participants in the Overall and Comorbid Sample, above and below cut-off scores of clinical significance 
(remission). 







Overall Sample 


TEG 




TG 




Chi Square 




Waitlist Control Group (In 
Treatment)' 




N 


% 


N 


% 


TGE vs. TG 




N 


% 


PHQ-9 


Pre-treatment score > 10/Total 


68/100 


68.0 


67/1 06 


63.2 


X 2 (1) 


= 0.52, P = 


.46 


22/42 


52.3 


Post-treatment score < 10 (Remission)/Total 


42/68 


61.7 


42/67 


62.6 


X 2 (1) 


= 0.11, P = 


.81 


14/22 


63.6 


3-month follow-up score < 10 (Remission)/Total 


51/68 


75.0 


46/67 


68.6 


X*(1J 


= .82, P = .48 


15/22 


68.1 


12-month follow-up score <10 (Remission)/Total 


44/68 


64.7 


46/67 


68.6 


X 2 (1) 


= 0.36, P = 


.62 


14/22 


63.6 


GAD-7 


Pre-treatment score > 8/Total 


76/100 


76.0 


74/106 


69.8 


X 2 (1) 


= 0.99, P = 


.31 


22/42 


52.3 


Post-treatment score < 8 (Remission)/Total 


47/76 


61.8 


46/74 


62.1 


X 2 (1) 


= 0.17, P = 


.76 


10/22 


45.5 


3-month follow-up score < 8 (Remission)/Total 


51/76 


67.1 


48/74 


64.8 


X 2 (1) 


= 0.22, P = 


.72 


15/22 


68.1 


12-month follow-up score <8 (Remission)/Total 


48/76 


63.1 


45/74 


60.8 


X 2 (l) 


= 0.29, P = 


.66 


14/22 


63.6 


Comorbid Sample 


Pre-treatment PHQ-9 score > 10 and GAD-7 
score > 8/Total 


60/100 


60.0 


54/1 06 


50.0 


X 2 (1) 


= 1.70, P = 


.19 


19/42 


45.2 


Post-treatment PHQ-9 score < 1 0 and GAD-7 
score <8 (Remission)/Total 


43/60 


71.6 


38/54 


70.3 


X 2 (1) 


= 0.09, P = 


.81 


14/19 


73.6 


3-month follow-up PHQ-9 score < 10 and 
GAD-7 score <8 (Remission)/Total 


51/60 


85.0 


39/54 


72.2 


X 2 (1) 


= 2.82, P = 


.16 


14/19 


73.6 


12-month follow-up PHQ-9 score < 10 and 
GAD-7 score <8 (Remission)/Total 


44/60 


73.0 


40/54 


74.0 


X 2 (1) 


= 0.23, P = 


.71 


14/19 


73.6 


Note. All data within table is pooled data from 20 generated sets of imputed data using the Multiple Imputation procedure. 
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emails facilitated more rapid symptom improvement with the 
Comorbid Sample who received automated emails relative to the 
Comorbid sample who did not receive emails. However, the 
results indicate that the automated emails were not absolutely 
necessary to achieve comparable outcomes in the longer term, as the 
TG group obtained similar outcomes as the TEG at the 12-month 
follow-up. This indicates that the decision about whether or not to 
include automated emails should be informed by the costs, risks 
and the benefits of their implementation and the value a service 
places on the speed of clinical improvements. Nevertheless, it is 
likely that more rapid reductions in symptoms would be associated 
with reduced health costs and lifetime burden of disease. 
Therefore, the inclusion of automated emails is recommended. 

These results also extend debate and questions about the 
mechanisms of change in internet interventions [31]. The 
magnitude of clinical benefits found in this study indicates that, 
at least for some patients, direct contact with a therapist is not 
necessary for clinically significant change to occur during internet 
treatment. This is not to say that therapeutic factors that facilitate 
engagement and adherence, such as the credibility of an 
intervention, reinforcement and unconditional positive regard, 
are not important. Rather, we contend that these and other factors 
can and should be integrated into the content, format and 
presentation of self-guided interventions. We suspect that as 
improvements are made to the content of internet-delivered 
psychotherapeutic interventions and in the delivery of that 
content, an increasing proportion of the population will be able 
to benefit from them. However, we caution against assuming that 
self-help materials can be easily developed and delivered. Just as 
therapists require careful training and supervised practice, internet 
intervention materials should also be carefully developed and 



improved through a process of repeated clinical evaluation and 
quality assurance cycles. 

We also contend that these results provide support for 
pragmatic models of psychotherapeutic change, which emphasise 
the importance of systematically learning, practicing and adopting 
core psychological skills. These skills support participants to 
change unhelpful habits of thought and action, which, in turn, 
have broad positive impacts on emotion dysregulation. We suspect 
the use of case-enhanced learning strategies (e.g., detailed case 
studies) in the course content facilitated engagement and 
adherence. However, as noted by others, multiple other factors 
such as providing a clear deadline and expectations of feedback 
also facilitate adherence and clinical outcomes [26]. It appears 
likely that systematic examination of other strategies to promote 
engagement and adherence in self-guided internet interventions, 
including prompts delivered by mobile technologies, will further 
improve outcomes in self-guided internet-delivered interventions 
and will also have utility across a broad range of health conditions 
[32]. 

Limitations 

These findings need to be considered in the context of the 
limitations of the overall study. The long-term effects of the study 
were not assessed alongside an active control group, although the 
large magnitude of the outcomes indicates there were strong 
treatment effects. The generalisability of these results is limited by 
the nature of the sample, which comprised people seeking internet- 
delivered treatment for anxiety and depression. However, it is 
important to note that a comparison of demographic and 
symptom profiles of participants in internet trials with those 
identified in epidemiological samples indicates that the results of 
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internet trials may generalise to the wider population [33]. 
Moreover, while symptom improvements were sustained, the 
present study relied on self-report measures rather than structured 
diagnostic interviews, which would have required clinical contact 
and may have affected outcomes. It is also important to note that 
the present study employed a specific transdiagnostic treatment 
intervention, the Wellbeing Course, which was designed and 
developed for use as either an entirely self-guided or clinician- 
guided Course. Consequently, it is possible that automated emails 
may facilitate greater adherence and clinical outcomes in less 
developed interventions or interventions initially developed to 
involve clinician guidance but subsequently used in a self-guided 
format. 

Conclusions 

The present results indicate that the benefits of the self-guided 
Wellbeing Course were sustained at 12-month follow-up. The 
inclusion of automated emails facilitated adherence, completion 
and more rapid symptom reduction during the Wellbeing Course 
in the immediate and short-term for those with comorbid 
symptoms. However, these emails did not improve outcomes any 
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